Overall performance metrics for multi-condition speaker recognition evaluations
نویسنده
چکیده
In this paper we propose a framework for measuring the overall performance of an automatic speaker recognition system using a set of trials of a heterogeneous evaluation such as NIST SRE2008, which combines several acoustic conditions in one evaluation. We do this by weighting trials of different conditions according to their relative proportion, and we derive expressions for the basic speaker recognition performance measures Cdet, Cllr, as well as the DET curve, from which EER and C det can be computed. Examples of pooling of conditions are shown on SRE-2008 data, including speaker sex and microphone type and speaking style.
منابع مشابه
Speaker recognition in a multi-speaker environment
We discuss the multi-speaker tasks of detection, tracking, and segmentation of speakers as included in recent NIST Speaker Recognition Evaluations. We consider how performance for the two-speaker detection task is related to that for the corresponding one-speaker task. We examine the effects of target speaker speech duration and the gender mix within test segments on results for these tasks. We...
متن کاملSpeakers In The Wild (SITW): The QUT Speaker Recognition System
This paper presents the QUT speaker recognition system, as a competing system in the Speakers In The Wild (SITW) speaker recognition challenge. Our proposed system achieved an overall ranking of second place, in the main core-core condition evaluations of the SITW challenge. This system uses an ivector/PLDA approach, with domain adaptation and a deep neural network (DNN) trained to provide feat...
متن کاملA note on performance metrics for Speaker Recognition using multiple conditions in an evaluation
In this paper we put forward arguments for pooling different evaluation conditions for calculating speaker recognition system performance measures. We propose a condition-based weighting of trials, and derive expressions for the basic speaker recognition performance measures Cdet, Cllr, as well as the DET curve, from which EER and C min det can be computed. We show that trials-based weighting i...
متن کاملSpeaker Adaptation in Continuous Speech Recognition Using MLLR-Based MAP Estimation
A variety of methods are used for speaker adaptation in speech recognition. In some techniques, such as MAP estimation, only the models with available training data are updated. Hence, large amounts of training data are required in order to have significant recognition improvements. In some others, such as MLLR, where several general transformations are applied to model clusters, the results ar...
متن کاملThe NIST speaker recognition evaluation - Overview, methodology, systems, results, perspective
This paper, based on three presentations made in 1998 at the RLA2C Workshop in Avignon, discusses the evaluation of speaker recognition systems from several perspectives. A general discussion of the speaker recognition task and the challenges and issues involved in its evaluation is oered. The NIST evaluations in this area and speci®cally the 1998 evaluation, its objectives, protocols and test...
متن کامل